it is known that in an electronic image pickup apparatus such as a digital camera, image blur due to hand movement or object movement is more likely to occur when a shutter speed is low. Mechanical hand movement correction and electronic hand movement correction may be employed as methods of suppressing image blur due to hand movement.
Mechanical hand movement correction may be performed using a lens shift method, a sensor shift method, and so on. In the lens shift method, image blur correction is performed by measuring a displacement amount using a gyro sensor or the like and driving a correction optical system for offsetting an imaging optical axis. In the sensor shift method, image blur correction is performed by moving an imaging device on the basis of a displacement amount measured using a similar gyro sensor or the like.
Meanwhile, in electronic hand movement correction, a plurality of images are captured at high speed and chronologically, and a positional displacement amount between the images is measured using a sensor or an image processing method. The positional displacement amount is then corrected, whereupon the plurality of images are added together to generate a non-blurry image.
A block matching method is known as a typical technique for determining the positional displacement amount between the images. In the block matching method, a block of an appropriate size (for example, 8 pixels×8 lines) is defined within a reference image, and a match index value is calculated within a fixed range from a corresponding location of a comparison image. The positional displacement amount between the images is then calculated by determining a position in which the calculated match index value is largest (or smallest depending on the index value).
The match index value may be a sum of squared intensity difference (SSD), which is a sum of squares of a pixel value difference, a sum of absolute intensity difference (SAD), which is a sum of absolute values of the pixel value difference, and so on. As SSD or SAD decreases, the match is determined to be closer. When pixel values of pixel positions pεI and qεI′ are set respectively as Lp, Lq in a reference block region I and a subject block region I′ of a matching operation, SSD and SAD are respectively expressed by the following Equations (1) and (2). It should be noted that p, q are quantities having two-dimensional values, I and I′ represent two-dimensional regions, and pεI indicates that a coordinate p is included in the region I.
                              SSD          ⁡                      (                          I              ,                              I                ′                                      )                          =                              ∑                                          p                ∈                I                            ,                              q                ∈                                  I                  ′                                                              ⁢                                    (                              Lp                -                Lq                            )                        2                                              (        1        )                                          SAD          ⁡                      (                          I              ,                              I                ′                                      )                          =                              ∑                                          p                ∈                I                            ,                              q                ∈                                  I                  ′                                                              ⁢                                                Lp              -              Lq                                                                      (        2        )            
A method employing a normalized cross-correlation (NCC) also exists. In a zero average correlation, average values Ave (Lp), Ave (Lq) of the pixels pεI and qεI′ included respectively in the reference block region I and the subject block region I′ of the matching operation are calculated. A difference between the pixel values included in the respective blocks is then calculated using the following Equations (3), (4).
                              Lp          ′                =                                            Lp              -                              Ave                ⁡                                  (                  Lp                  )                                                                                                      1                  n                                ⁢                                                      ∑                                          p                      ∈                      I                                                        ⁢                                                            (                                              Lp                        -                                                  Ave                          ⁡                                                      (                            Lp                            )                                                                                              )                                        2                                                                                ⁢                      |                          p              ∈              I                                                          (        3        )                                          Lq          ′                =                                            Lq              -                              Ave                ⁡                                  (                  Lq                  )                                                                                                      1                  n                                ⁢                                                      ∑                                          q                      ∈                                              I                        ′                                                                              ⁢                                                            (                                              Lq                        -                                                  Ave                          ⁡                                                      (                            Lq                            )                                                                                              )                                        2                                                                                ⁢                      |                          q              ∈                              I                ′                                                                        (        4        )            
Next, a normalization cross-correlation NCC is calculated using Equation (5).NCC=ΣLp′Lq′  (5)
Blocks having a large normalization cross-correlation NCC are determined to be a close match (have a high correlation), and the relative displacement amount between the blocks I′ and I exhibiting the closest match is determined.
For example, in a case where movement of a hand of a photographer is corrected using electronic blur correction, it is important to determine the movement of the entire image with stability when determining the inter-image positional displacement amount using the method described above.
When an object included in a photographed image is stationary, movement within individual regions matches the movement of the entire image. Therefore, in the matching processing described above, a motion vector calculation result from a suitable region for determining the image displacement amount may be regarded as the movement of the entire image. Normally, a motion vector is determined in each of a plurality of regions of the image, whereupon a highly reliable motion vector, which is obtained by excluding the motion vectors of regions that are unsuitable for determining a motion vector, such as noisy regions, low contrast regions, and repeated pattern regions, is selected as the movement of the entire image.
On the other hand, when the object included in the photographed image includes movement, the overall movement of the image must be distinguished from the movement of the object before the motion vector of the entire image is selected. In JP8-251474A, for example, an object is divided into a plurality of regions, a region suitable for determining a motion vector is selected using the sizes of the motion vectors of the respective regions, the sizes of the regions themselves, and so on as a reference, and the motion vector of the selected region is set as the overall movement.
Here, distortion normally exists in an optical system. In particular, when photography is performed using a lens having a large distortion such as a wide angle lens, the distortion is small in a central portion of the image, and therefore an image having large distortion on the image end portions is obtained (see FIGS. 16A and 16B). When the inter-image positional displacement amount is determined using an image having this type of distortion, unevenness in the motion vector occurs in accordance with a distance from the center of the image (to be referred to hereafter as the image height). As shown in FIG. 16A, in an image photographed using an optical system having negative distortion, the motion vector of a region having a great image height is smaller than the motion vector of a region having a small image height. Conversely, as shown in FIG. 16B, in an image photographed using an optical system having positive distortion, the motion vector of a region having a great image height is greater than the motion vector of a region having a small image height.